Iteration complexity of randomized block-coordinate descent methods for minimizing a composite function

Iteration complexity of randomized block-coordinate descent methods for minimizing a composite function

In this paper we develop a randomized block-coordinate descent method for minimizing the sum of a smooth and a simple nonsmooth block-separable convex function and prove that it obtains an ε-accurate solution with probability at least 1− ρ in at most O((n/ε) log(1/ρ)) iterations, where n is the number of blocks. This extends recent results of Nesterov [Efficiency of coordinate descent methods o...

On the Complexity Analysis of Randomized Block-Coordinate Descent Methods

In this paper we analyze the randomized block-coordinate descent (RBCD) methods proposed in [11, 15] for minimizing the sum of a smooth convex function and a blockseparable convex function, and derive improved bounds on their convergence rates. In particular, we extend Nesterov’s technique developed in [11] for analyzing the RBCD method for minimizing a smooth convex function over a block-separ...

Iteration Complexity of Feasible Descent Methods Iteration Complexity of Feasible Descent Methods for Convex Optimization

In many machine learning problems such as the dual form of SVM, the objective function to be minimized is convex but not strongly convex. This fact causes difficulties in obtaining the complexity of some commonly used optimization algorithms. In this paper, we proved the global linear convergence on a wide range of algorithms when they are applied to some non-strongly convex problems. In partic...

Block coordinate descent methods for semidefinite programming

Improved Iteration Complexity Bounds of Cyclic Block Coordinate Descent for Convex Problems

The iteration complexity of the block-coordinate descent (BCD) type algorithm has been under extensive investigation. It was recently shown that for convex problems the classical cyclic BCGD (block coordinate gradient descent) achieves an O(1/r) complexity (r is the number of passes of all blocks). However, such bounds are at least linearly depend on K (the number of variable blocks), and are a...